A/B Split Testing and Optimization

A/B split testing allows you to test different mailing characteristics in an effort to produce higher response rates. This type of testing gives you the opportunity to gain insight into which content and subject lines yield the best results across your subscriber base.

 

In this topic::

 

When you use PostUp’s A/B split testing and optimization functionality, you have the option to:

  • Test up to 8 different subject lines, preview text, friendly from names and/or elements of your content
  • Reserve between 1% and 100% of the selected mailing list for testing purposes
  •  Automatically send the winning combination, tested on a sample set, to the remainder of the mailing list after a specified time period. PostUp determines the winning combination based on one of the following:
    • Total Opens
    • Total Clicks
    • Shares and Likes
    • Referred Clicks
    • Conversion Rate

Note: Your account settings determine your level of system access. If several of the options and/ or processes outlined in this document are not available to you, contact your site administrator.

 

Use Case Scenario

Say you have two pieces of email creative. Which one will your audience respond better to?

Following the steps outlined below, you can easily create an A/B split mailing that tests the two different content combinations.

You may select to test the two pieces of creative using a sample set of your mailing list, say 20% (10% will receive creative A and 10% will receive creative B) and enable the auto-optimization feature. Once enabled, PostUp will automatically send the winning version of the sample test to the remaining recipients (80%), after the selected time period based on opens, clicks, shares, likes, referred clicks or conversion rate.

Back to top

 

Building an A/B Split Mailing

  1. Once logged in, click on the Send tab.
  2. Create or edit a mailing, per normal procedure.
  3. Click on the Content tab.

  4. Click on the checkbox next to A/B Split Mailing, located under Headers section. The A/B Split Characteristics section will appear above the Headers section.
  5. Select the percentage of recipients to reserve for testing from the first dropdown menu.
  6. Select the number of testing combinations from the second dropdown menu.

    7. Note: Steps 7 through 9 are optional. You may select to follow these steps if you want to enable the auto-optimization feature. The PostUp Professional Services team highly recommends that you utilize this feature.

  7. [Optional] Click on the checkbox to automatically send the winning combination to the remaining recipients.
  8. [Optional] Select the metric for determining the winning combination (opens, clicks, shares, likes or referred clicks) from the first dropdown menu.
  9. [Optional] Select the evaluation time frame from the second dropdown menu.

Based on the selections made in the example above, if you reserved 10% of your mailing list to test two different combinations: 5% will get the default combo, 5% will get combo 2. PostUp will evaluate the number of opens for a 2 hour period. After 2 hours, PostUp will take the combination with the highest number of opens and send it to the remaining 90% of your list.

Edit the Mailing Combinations

  1. Enter the default subject line and content for the Default Combo.
  2. Click on the Combo 2 tab.
  3. Click on Edit to update the appropriate field with the new variable.

  4. Repeat, if testing multiple variables.
  5. Click Apply Changes.
  6. Repeat steps 2 through 5 to update additional testing combinations.
  7. Schedule the mailing per normal procedure.

Note: In order to send a test message for each combination, users need to activate and send a test message from each Combo tab.

Reporting

A/B split mailings will appear under the Mailing summary section on the Reports tab in parent/ child groupings. Testing combinations will appear with a symbol (such as #1, #2, W) in front of the mailing title. You may download the Detailed Mailing Report for each of the child mailings, as well as for the winning combination mailing.

Back to top